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Abstract 

A  knowledge  acquisition  tool  should  provide  a  user  with  maximum  guidance  in  extending 
and  debugging  a  knowledge  base,  by  preventing  inconsistencies  and  knowledge  gaps  that 
may  arise  inadvertently.  Most  current  acquisition  tools  are  not  very  flexible  in  that  they 
are  built  for  a  predetermined  inference  structure  or  problem-solving  mechanism,  and  the 
guidance  they  provide  is  specific  to  that  inference  structure  and  hard-coded  by  their  designer. 
This  paper  focuses  on  EXPECT,  a  reflective  architecture  that  supports  knowledge  acquisition 
based  on  an  explicit  analysis  of  the  structure  of  a  knowledge-based  system,  rather  than  on 
a  fixed  set  of  acquisition  guidelines.  EXPECT’s  problem  solver  is  tightly  integrated  with 
LOOM,  a  state-of-the-art  knowledge  representation  system.  Domain  facts  and  goals  are 
represented  declaratively,  and  the  problem  solver  keeps  records  of  their  functionality  within 
the  task  domain.  When  the  user  corrects  the  system’s  knowledge,  EXPECT  tracks  any  possible 
implications  of  this  change  in  the  overall  system  and  cooperates  with  the  user  to  correct  any 
potential  problems  that  may  arise.  The  key  to  the  flexibility  of  this  knowledge  acquisition 
tool  is  that  it  adapts  its  guidance  as  the  knowledge  bases  evolve  in  response  to  changes 
introduced  by  the  user. 

Keywords:  knowledge  acquisition;  knowledge-base  refinement;  intelligent  architectures. 


1  Introduction 


The  knowledge  about  a  task  is  not  a  collection  of  isolated  information  packets  but  rather  a 
carefully  constructed  web  of  facts,  data,  and  procedures.  The  details  of  how  knowledge  is 
organized  and  how  it  interacts  may  be  unknown  to  the  user  and  often  hard  to  keep  track  of. 
The  key  to  knowledge  acquisition  is  thus  not  in  supporting  the  addition  of  more  items  to  the 
collection,  but  in  ensuring  harmonious  interactions  between  new  and  existing  knowledge  and 
preventing  redundancies,  inconsistencies,  and  knowledge  gaps  that  may  arise  inadvertently. 

Most  tools  for  knowledge  acquisition  achieve  this  by  having  expectations  about  how 
each  piece  of  knowledge  fits  in  the  overall  system  [Marcus  and  McDermott,  1989;  Musen, 
1989;  Kahn  et  a/.,  1985].  For  example,  systems  for  classification  tasks  need  knowledge  for 
mapping  inputs  into  classes.  When  the  user  enters  a  new  class,  their  acquisition  tools  always 
expect  to  be  given  knowledge  about  how  to  assign  an  input  to  that  new  class.  Having 
these  expectations  is  very  useful  to  support  knowledge  acquisition  in  that  they  allow  the 
system  to  ensure  that  changes  are  introduced  by  the  user  in  a  harmonious  way.  However, 
since  each  tool  is  designed  for  a  specific  type  of  task,  the  expectations  are  hard-coded  in  the 
tool.  This  limits  a  tool’s  flexibility,  since  applications  often  do  not  conform  exactly  to  the 
type  of  task  that  a  tool  was  designed  for  and  multiple  problem-solving  approaches  may  be 
required  within  a  given  application.  In  addition,  it  is  hard  to  determine  beforehand  the  type 
of  problem-solving  method  that  is  needed  for  a  new  application. 

The  goal  of  the  EXPECT  project  is  to  build  tools  for  knowledge  refinement  that  are 
both  flexible  and  supportive  to  the  user.  EXPECT  forms  dynamic  expectations  based  on  the 
current  knowledge  of  the  system  by  understanding  the  content  of  the  knowledge  bases  and 
their  interactions.  These  expectations  are  not  hard-coded  in  the  knowledge  acquisition  tool; 
instead,  they  are  explicitly  created  by  the  system  when  needed.  The  key  to  this  goal  is  a 
reflective  architecture,  i.e.,  a  system  that  has  the  ability  to  introspect  and  determine  exactly 
what  every  piece  of  knowledge,  old  and  new,  contributes  to  the  task. 

EXPECT’s  architecture  is  reflective  because  it  represents  and  manipulates  many  different 
types  of  knowledge  distinctly  and  explicitly.  Factual  domain  knowledge  about  a  task  (e.g.,  de¬ 
scriptions,  relations,  and  definitions)  is  represented  in  LOOM  [MacGregor,  1988;  MacGregor, 
1991],  a  state-of-the-art  knowledge  representation  system  of  the  KL-ONE  family.  Problem¬ 
solving  knowledge  is  represented  in  a  procedural-style  language  that  allows  subgoal  posting, 
control  programming  constructs,  and  expressive  parameter  typing.  EXPECT  captures  the 
semantics  of  goals  by  translating  them  into  LOOM  concepts.  The  problem  solver  uses  this 
representation  to  reason  about  actions  and  their  relation  to  concepts  and  instances  in  the 
domain.  By  representing  each  type  of  knowledge  declaratively  and  supporting  interrelation¬ 
ships  between  them,  EXPECT  has  access  to  better  understanding  about  the  task  than  other 
architectures  may  have. 

This  paper  begins  with  an  overview  of  EXPECT’s  architecture  and  its  knowledge  acqui¬ 
sition  tool.  Next  we  describe  how  a  user  interacts  with  the  knowledge  acquisition  tool  to 
change  the  knowledge  initially  given  to  the  system,  taking  examples  from  an  application 
that  evaluates  transportation  plans.  The  paper  continues  with  a  discussion  of  how  EXPECT 
relates  to  other  relevant  work  on  knowledge  acquisition.  Finally,  we  present  our  plans  for 
future  work  and  a  conclusion. 
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Figure  1:  A  schematic  representation  of  EXPECT’s  architecture. 


2  The  expect  Architecture 

EXPECT  builds  on  previous  research  on  the  Explainable  Expert  System  (EES)  project  [Neches 
et  al.,  1985;  Swartout  et  al. ,  1991].  EXPECT’s  architecture  is  designed  to  provide  an  under¬ 
standing  of  how  each  piece  of  information  in  a  knowledge- based  system  contributes  to  solving 
a  task.  This  understanding  comes  from  various  features  of  the  architecture  that  we  describe 
briefly  in  this  section. 

Figure  1  shows  an  overview  of  EXPECT’s  architecture.  In  EXPECT,  any  information  neces¬ 
sary  to  perform  a  task  is  represented  distinctly  according  to  its  nature  either  as  domain  facts 
or  as  problem-solving  knowledge.  Domain  facts  are  represented  in  LOOM  [MacGregor,  1988; 
MacGregor,  1991].  LOOM  provides  a  descriptive  logic  representation  language  and  includes 
a  classifier  for  inference.  Problem-solving  knowledge  is  expressed  as  EXPECT’s  methods.  A 
method  in  EXPECT  is  an  abstract  and  generic  description  of  how  a  goal  can  be  achieved 
including  the  goal,  a  method  body  that  describes  the  procedure  to  achieve  that  goal,  and 
the  result  that  the  method  is  expected  to  return.  The  goal  of  each  method  is  also  represented 
in  LOOM,  referring  to  the  concepts  and  instances  that  appear  in  the  parameter  list. 

Figure  2  shows  an  example  of  EXPECT’s  representation  of  factual  and  problem-solving 
knowledge  in  a  transportation  domain.  The  first  expression  specifies  that  seaport  is  a  kind 
of  port  with  ships,  berths,  covered  storage  area,  and  piers.  The  second  expression  is  a  very 
simple  method  to  find  the  seaports  of  a  location  by  retrieving  the  value  of  the  r- seaports 
relation  of  the  location.  The  last  expression  in  the  figure  is  a  more  complex  method  to 
determine  whether  a  ship  fits  in  a  seaport.  Beside  relations,  the  method  body  can  contain 
subgoals  which  can  be  combined  using  control  structures  such  as  conditional  and  iteration 
statements.  The  methods  shown  here  are  domain-specific,  but  domain-independent  generic 
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(dal concept  SEAPORT 
: ia-priaitive  port 

: constraints  (and  (sons  r-sbips  ship) 

(sons  r-berths  berth) 

(some  r-covered-storage-axea  number) 

(some  r-piers  number))) 

(delmethod  FIID-SEAPORTS 

:goal  (find  (obj  ?s  is  (set-ot  (inst-ol  seaport))) 

(ol  (?loc  is  (inst-ol  location)))) 

:result  (set-ol  (inst-ol  seaport)) 

:method-body  (r- seaports  ?loc)) 

(delmethod  D ETERMIIE-VH ETHER-SHIP- FITS- I J-SE1 PORT 
.‘goal  (determ  ine-shether 

(obj  (?ship  is  (inst-ol  ship))) 

(lits-in  (?port  is  (inst-ol  seaport)))) 

: result  (inst-ol  boolean) 

:method-body  (less  (obj  (r-length  ?ship)) 

(than  (compute-max  (obj  (spec-ol  length)) 

(ol  (set-ol  (spec-ol  berth))) 
(in  Tport))))) 

Figure  2:  Factual  and  problem-solving  knowledge  in  EXPECT. 


methods  can  be  expressed  with  this  same  language. 

In  order  to  ensure  coherence  among  the  various  types  of  knowledge,  the  problem  solver 
uses  the  factual  and  problem-solving  knowledge  sources  to  perform  a  static  analysis  of  a 
given  top-level  goal,  recording  how  each  piece  of  knowledge  contributes  to  the  problem¬ 
solving  process.  EXPECT  employs  the  reasoning  capabilities  of  LOOM  augmented  with  goal 
refinement  and  reformulation  as  follows.  EXPECT’s  analysis  is  effectively  a  partial  evaluation 
of  the  given  top-level  goal.  If  no  method  is  found  to  achieve  a  posted  goal,  the  goal  is 
reformulated  using  the  factual  domain  knowledge  into  a  set  of  subgoals  that  can  be  achieved. 
For  example,  if  there  is  no  method  to  find  the  speed  of  a  ship  and  three  types  of  ships  are 
described  in  the  domain  knowledge,  the  system  will  reformulate  this  goal  into  three  subgoals 
and  look  for  a  method  to  find  the  speed  of  each  of  the  three  types  of  ship.  This  analysis 
provides  the  knowledge  acquisition  tool  with  an  understanding  of  both  the  functionality  and 
the  nature  of  all  the  information  used  for  the  task. 

Another  important  source  of  understanding  is  the  tight  integration  of  the  problem  solver 
with  the  LOOM  classifier.  In  addition  to  classes  find  instances,  EXPECT  represents  in  LOOM 
all  the  goals  that  arise  during  problem  solving  and  matches  goals  and  methods  based  on 
their  semantics  using  loom’s  classifier.  EXPECT’s  matcher  can  find  methods  to  achieve  a 
posted  goal  taking  into  account  the  semantic  definitions  of  their  respective  arguments.  Any 
goal  (i.e.,  a  posted  subgoal  or  a  goal  that  a  method  achieves)  is  represented  as  a  concept.  For 
example,  the  LOOM  definition  of  the  goal  achieved  by  the  method  find- seaports  (shown  in 
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Figure  2)  is: 


(def concept  goal-concept — f ind- seaport s 
:is  (and  goal-concept  FIND 
(the  OBJ 

(and  instance-set  seaport)) 

(the  OF 

(and  instance-description  location)))) 

The  matcher  finds  the  type  of  the  most  general  bindings  for  variables  that  can  be  used 
to  unify  a  posted  goal  and  the  goal  that  a  method  can  achieve.  The  goal’s  arguments  may 
be  given  in  any  order,  because  the  matcher  maps  arguments  according  to  their  names. 

In  EXPECT,  problem-solving  errors  also  provide  useful  information  for  knowledge  acqui¬ 
sition.  Errors  (or  simply  anything  that  the  system  is  not  sure  about  and  would  like  the 
user  to  check)  may  come  from  the  parser,  the  matcher,  or  the  static  analyzer.  Instead  of 
interrupting  problem  solving  when  an  error  arises,  the  system  takes  it  as  a  need  for  the 
knowledge  acquisition  module  to  request  the  user’s  intervention.  The  problem  solver  reasons 
about  these  errors  and  provides  detailed  information  to  the  knowledge  acquisition  tool  that 
is  crucial  to  support  the  user  in  correcting  them,  as  we  show  in  the  next  section. 

In  summary,  the  facility  to  relate  all  the  different  sources  of  knowledge  in  the  system  and 
capture  their  influence  in  the  system’s  behavior  enables  EXPECT ’s  knowledge  acquisition  tool 
to  support  the  user  in  changing  the  system’s  knowledge. 


3  Knowledge  Acquisition  in  EXPECT 

EXPECT’s  knowledge  acquisition  module  is  invoked  any  time  that  errors  in  the  knowledge 
bases  are  encountered  during  problem  solving.  Besides  errors,  the  problem  solver  also  signals 
possible  problems  that  may  result  from  a  user’s  changes  to  the  knowledge  base.  The  user 
may  not  always  foresee  these  possible  problems,  so  the  system  takes  the  responsibility  to 
track  them.  We  consider  them  possible  lapses  on  the  part  of  the  user,  and  they  are  added 
to  an  agenda  of  items  that  require  user  intervention.  For  example,  if  the  user  adds  a  new 
method  that  achieves  the  same  goal  as  a  method  that  already  exists,  the  system  detects  this 
and  notifies  the  user.  Lapses  are  not  necessarily  errors,  often  they  reflect  things  that  the 
system  brings  to  the  user’s  attention  and  can  be  dismissed  by  the  user  after  giving  them 
consideration.  The  system  bases  the  agenda’s  requests  on  information  that  is  actually  needed 
for  problem  solving,  and  as  a  result  the  user  is  not  bothered  with  unnecessary  interventions. 

EXPECT  has  a  catalog  of  possible  types  of  lapses  together  with  possible  actions  that  the 
user  may  be  suggested  to  take  in  order  to  correct  each  type  of  lapse.  The  information  about 
lapses  is  explicitly  represented,  and  we  find  it  is  very  useful  as  a  means  for  the  problem 
solver  to  provide  feedback  to  the  knowledge  acquisition  tool  regarding  the  status  of  the 
current  knowledge  base. 

Figure  3  shows  a  snapshot  of  EXPECT’s  user  interface.  It  allows  the  user  to  examine  the 
content  of  the  knowledge  bases  and  navigate  through  problem-solving  episodes  to  understand 
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Figure  3:  EXPECT’s  user  interface. 


how  the  system  achieves  the  task  goals.  The  left  side  of  the  screen  is  showing  the  problem¬ 
solving  tree  and  the  right  side  the  current  items  pending  on  the  agenda.  In  this  snapshot, 
the  user  had  asked  for  a  description  of  the  instance  Los  Angeles,  which  caused  the  smaller 
window  in  the  lower  right  corner  to  pop  up.  In  the  description  of  Los  Angeles,  the  system 
indicates  that  it  is  a  location  and  that  additional  information  about  the  seaports  of  Los 
Angeles  is  needed  in  order  to  use  this  instance  for  problem  solving.  EXPECT  knows  that 
this  is  needed  because  it  is  used  in  the  method  to  find  the  seaports  of  a  location  shown  in 
Figure  2.  Notice  that  this  request  is  also  an  item  on  the  agenda  (the  second  one). 

The  user  can  interact  with  the  system  to  resolve  items  on  the  agenda,  or  take  the  initiative 
to  add  new  knowledge  or  change  the  knowledge  already  in  the  system.  EXPECT  analyzes 
how  any  change  introduced  by  the  user  affects  the  problem  solving  required  for  the  task, 
and  tries  to  detect  any  negative  consequences  provoked  by  the  change.  These  also  raise 
lapses  that  axe  included  in  the  user’s  agenda.  The  knowledge  acquisition  tool  can  be  used  to 
correct  and  extend  the  problem-solving  knowledge  as  well  as  the  factual  knowledge.  Table  1 
summarizes  the  analysis  that  EXPECT  performs  for  every  modification  done  by  the  user.  The 
next  sections  describe  with  examples  how  this  analysis  supports  the  knowledge  acquisition 
process. 
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When  the  user  adds  an  instance  of  a  type: 


1.  If  the  type  given  for  the  instance  is  more  general  than  the  types  that  are  used  for  problem 
solving,  find  more  specific  types  in  the  knowledge  base  and  ask  user  to  choose  one. 

2.  Look  up  the  roles  defined  for  the  instance  type.  Of  those,  find  which  roles  are  used  for 
problem  solving  and  ask  the  user  for  their  value. 

When  the  user  changes  any  method: 

1.  If  the  new  method  uses  a  role  of  a  type  that  it  did  not  use  before,  ask  user  for  the  value  of 
that  role  for  ail  the  instances  of  that  type. 

2.  If  the  new  method  posts  a  subgoal,  check  that  there  is  a  method  that  matches  with  that 
subgoal.  If  not,  notify  the  user. 

3.  If  the  new  method  is  not  used  to  achieve  any  subgoal,  notify  the  user. 

4.  If  the  new  method  has  syntactic  errors,  ask  user  for  corrections. 

5.  If  the  new  method  contains  information  that  is  not  used  for  problem  solving,  notify  the  user. 


Table  1:  EXPECT  performs  a  thorough  analysis  of  every  modification  done  by  the  user. 


4  Adding  New  Instances 

Suppose  that  the  user  wants  to  add  a  new  instance.  He  or  she  selects  the  menu  to  add  a 
new  instance,  and  types: 

Name:  Long  Beach 
Type:  port 

When  a  new  instance  is  defined,  EXPECT  checks  that  the  type  given  is  specific  enough.  In 
this  case,  the  type  is  port,  whose  subtypes  are  airport  and  seaport.  The  problem  solver’s 
analysis  of  the  task  shows  that  the  system  needs  to  know  the  ships  available  at  a  seaport. 
This  is  an  indication  for  the  knowledge  acquisition  tool  that  it  is  important  to  know  if  Long 
Beach  is  a  seaport.  Thus,  EXPECT  requires  the  user  to  be  more  specific  about  the  type  of 
port  that  Long  Beach  is.  The  user  is  shown  the  subtypes  of  port  and  is  asked  to  pick  among 
them.  If  the  user  does  not  know  this  information,  EXPECT  will  accept  port,  but  will  place  an 
item  on  the  agenda  to  remind  the  user  to  provide  this  information  when  it  becomes  available. 
But  let  us  suppose  that  the  user  picks  seaport  as  the  type  of  the  Long  Beach  port. 

EXPECT  next  checks  what  information  is  needed  for  problem  solving  about  this  type  of 
instance.  One  method  uses  the  ships  available  at  a  seaport  and  its  berths.  So  the  system 
asks  the  user  for  this  information.  Again,  if  this  information  is  not  available,  EXPECT  will 
place  these  requests  in  the  agenda. 


Now  suppose  that  the  user  adds  a  new  instance  called  Los  Angeles  of  type  location. 
This  type  is  specific  enough  for  the  problem-solving  methods,  and  the  information  that  they 
need  about  locations  is  what  seaports  they  have.  EXPECT  includes  a  new  item  in  the  agenda 
to  request  this  information. 

At  this  point  the  agenda  contains  several  items,  each  representing  requests  for  information 
about  the  new  instances  just  defined.  EXPECT  provides  the  user  with  specific  support  for  each 
type  of  item  on  the  agenda.  Consider  the  item  requesting  information  about  the  seaports  of 
Los  Angeles.  If  the  user  clicks  on  this  item,  EXPECT  pops  up  a  menu  with  an  explanation 
of  why  this  information  is  needed  (i.e.,  which  methods  use  this  and  what  they  accomplish). 
The  menu  also  contains  possible  actions  that  the  user  can  undertake  to  resolve  this  agenda 
item.  In  this  case,  the  item  requests  the  value  of  a  role  for  an  instance.  EXPECT  suggests 
that  the  user: 

•  provide  a  value 

•  remove  the  instance 

•  modify  the  method,  so  it  will  not  need  this  information. 

Based  on  its  underlying  knowledge,  EXPECT  prescribes  different  solutions  for  each  type 
of  item  in  the  agenda.  This  information  is  represented  declaratively,  and  EXPECT  uses  it  to 
dynamically  create  suggestions  that  are  specific  to  the  agenda  item.  In  this  case,  if  the  user 
chooses  to  provide  the  value  Long  Beach,  then  the  agenda  item  will  disappear. 

This  knowledge  acquisition  dialogue  contains  several  important  points.  First,  the  system 
understands  how  each  type  of  instance  is  used  and  provides  support  for  knowledge  acquisition 
based  on  this  understanding.  If  the  methods  change  and  new  information  about  a  type  of 
instance  is  required,  the  system  will  realize  this  and  update  the  agenda  accordingly.  Second, 
the  user  is  insulated  from  the  details  of  the  implementation  because  EXPECT  keeps  track  of 
what  information  is  needed.  Third,  the  agenda  makes  the  dialogue  very  flexible,  since  the 
user  chooses  when  to  attend  to  an  item  raised  by  the  system. 

5  Adding  New  Problem-Solving  Knowledge 

EXPECT  also  allows  a  user  to  modify  problem-solving  knowledge.  The  user  can  add  more 
detail  to  aun  existing  method  by  inserting  steps  in  the  method  body,  or  change  a  method’s 
goal  by  adding  new  parameters  or  modifying  the  types  of  the  ones  that  it  currently  has.  The 
user  can  also  add  new  methods  as  we  describe  next. 

If  the  user  creates  the  method  from  scratch,  EXPECT  can  provide  little  help  because  it 
will  not  have  an  understanding  of  this  new  method  until  the  user  is  finished  with  it.  Instead, 
EXPECT  encourages  the  user  to  re-use  existing  methods  in  the  knowledge  base  by  using  them 
as  the  basis  for  new  methods.  Because  EXPECT  already  understands  these  methods,  it  can 
provide  help  in  adapting  them  to  new  uses. 

For  example,  suppose  that  the  user  wants  to  add  a  method  to  find  the  airports  of  a 
location.  In  this  case  the  user  indicates  that  the  new  method  to  find  airports  is  similar  to 
the  existing  method  to  find  seaports  (the  one  shown  in  Figure  2).  EXPECT  uses  the  latter  as 
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a  model,  counting  on  the  user  to  specify  all  differences.  The  user  indicates  that  the  relation 
r-seaports  has  to  be  changed  by  r-airports,  and  the  concept  seaport  by  airport. 

EXPECT  builds  a  new  method  with  these  changes  and  adds  it  to  the  problem-solving 
knowledge  base.  Then  it  checks  how  the  new  method  affects  the  rest  of  the  knowledge 
currently  in  the  system.  The  same  checks  Me  performed  if  an  existing  method  is  modified. 

The  first  check  concerns  the  validity  of  the  new  method  in  itself.  This  includes  looking 
for  syntax  errors,  inconsistent  type  passing,  and  steps  in  the  method  whose  results  are  not 
used.  Each  lapse  detected  becomes  an  item  on  the  agenda. 

The  second  matter  that  EXPECT  checks  is  how  the  new  method  relates  to  the  rest  of 
the  methods.  EXPECT  understands  that  methods  Me  used  to  achieve  goals,  so  it  will  run 
the  static  analyzer  and  make  sure  that  the  method  is  useful  and  that  the  subgoals  that  the 
method  body  contains  can  be  achieved  by  other  methods.  Again,  EXPECT  warns  the  user 
via  the  agenda  if  it  finds  any  problems. 

Last,  EXPECT  checks  if  the  new  method  needs  information  about  instances  that  is  not 
currently  available.  In  this  case,  it  realizes  that  find-airports  retrieves  the  value  of 
r-airports  of  instances  of  type  location  and  consequently  it  adds  an  item  to  the  agenda 
to  request  this  value  for  Los  Angeles. 

The  scenario  just  described  cleMly  follows  an  analogy  process.  EXPECT  provides  a  frame¬ 
work  for  analogical  reasoning  where  the  user  suggests  the  source  of  the  analogies,  the  map¬ 
ping,  and  any  necessary  adaptations,  and  the  tool  provides  the  supporting  environment  for 
navigating  through  the  system’s  reasoning  and  cMrying  out  the  user’s  corrections  through 
analogical  reasoning  or  any  other  mechanisms.  We  Me  currently  extending  our  system  to 
provide  support  for  finding  similM  methods  using  a  non-exact  matcher  as  we  discuss  below. 
It  is  important  to  point  out  that  if  the  user  leaves  out  anything  that  is  relevant,  the  analysis 
that  EXPECT  performs  to  check  the  validity  of  the  new  method  may  detect  that  this  is  the 
case  and  raise  agenda  items  to  be  resolved  by  the  user. 

6  Related  Work 

EXPECT’8  reflective  Mchitecture  provides  an  understanding  of  the  knowledge  in  the  system 
that  can  be  used  to  form  expectations  about  any  new  knowledge  being  added.  Having  ex¬ 
pectations  is  be  a  powerful  basis  to  support  knowledge  acquisition.  TEIRESIAS  [Davis,  1980] 
used  statistical  techniques  to  form  expectations  about  what  terms  were  likely  to  co-occur. 
More  recent  tools,  such  as  SALT  [Mmcus  and  McDermott,  1989],  PROTEGE  [Musen,  1989], 
and  MORE  [Kahn  et  al .,  1985],  are  built  for  a  specific  inference  structure  (e.g.,  classifica¬ 
tion)  and  expect  their  knowledge  base  to  be  populated  with  information  useful  for  that  type 
of  task  [ChandrasekMan,  1986;  McDermott,  1988].  However,  since  their  expectations  Me 
hMd-coded,  these  tools  do  not  provide  much  flexibility  [Musen,  1992].  The  problem-solving 
structure  of  an  application  cannot  always  be  defined  in  domain-independent  terms.  Further¬ 
more,  these  method-specific  inference  mechanisms  may  not  address  some  of  the  pMticulars 
of  an  application  simply  because  they  were  designed  with  generality  in  mind.  Another  prob¬ 
lem  with  the  method-specific  knowledge  acquisition  tools  is  that  they  raise  the  non-trivial 
issue  of  determining  a  library  of  possible  methods.  The  work  involved  in  handcrafting  such  a 
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library  of  methods,  making  sure  to  both  provide  wide-coverage  of  tasks  and  well-understood 
characterizations  of  the  inference  capabilities  of  each  method,  is  daunting. 

To  address  these  limitations,  some  researchers  [Klinker  et  al,  1991;  Puerta  et  ah ,  1992]  are 
developing  libraries  of  problem-solving  methods  that  handle  finer-grained  inference  structures 
than  the  ones  above.  These  approaches  provide  more  flexibility  in  building  a  knowledge- 
based  system,  and  we  share  their  belief  that  this  is  a  step  in  the  right  direction.  EXPECT’s 
expectations  are  as  fine-grained  as  the  user’s  definitions.  They  are  not  hard  coded,  and  are 
based  on  understanding  each  piece  of  knowledge  both  individually  and  in  conjunction  with 
others.  EXPECT  can  be  applied  to  tasks  with  any  kind  of  inference  structure. 

NEODISCIPLE  [Tecuci,  1992]  integrates  several  machine  learning  techniques  in  a  knowl¬ 
edge  acquisition  tool.  NEODlSCIPLE  takes  a  user-given  answer  to  a  problem  and  applies 
explanation-based  learning  to  build  a  plausible  proof  tree,  abduction  to  complete  the  proof, 
and  several  other  learning  techniques  to  generalize  the  proof.  Its  predecessor,  DISCIPLE 
[Tecuci  and  Kodratoff,  1990],  built  an  analogy  with  an  existing  proof  when  the  system  lacked 
domain  knowledge  to  build  the  proof  for  a  new  input.  Our  approach  automates  different 
parts  of  the  analogical  process.  The  user  suggests  the  source  of  the  analogies,  the  mapping, 
and  any  necessary  adaptations.  Our  tool  provides  support  for  carrying  it  out,  checking  the 
validity  of  the  new  knowledge,  and  examining  its  effects  in  the  current  knowledge  bases. 

7  Discussion 

We  plan  to  extend  EXPECT’s  reflectiveness  in  two  main  directions.  One  is  to  improve  the 
understanding  of  goals  through  a  relaxed  semantic  matcher,  and  the  other  is  to  extend 
the  current  representation  of  agenda  items  to  provide  more  comprehensive  support  for  the 
knowledge  acquisition  tool. 

We  have  extended  EXPECT’s  semantic  matcher  to  find  non-exact  matches  of  methods  and 
goals.  By  dropping  parts  of  the  definition  of  the  goals,  this  relaxed  matcher  effectively  does 
a  partial  unification.  The  relaxed  matcher  may  propose  a  method  that  has  five  parameters 
that  match  exactly  five  of  the  six  parameters  of  a  posted  goal.  If  a  goal’s  parameter  is  of  a 
certain  type  and  no  methods  are  found  that  match  exactly,  the  relaxed  matcher  may  propose 
a  method  that  applies  to  a  more  specific  type  or  to  another  subtype  of  the  same  direct 
supertype.  The  relaxed  matcher  can  also  describe  what  relaxations  yielded  the  retrieved 
method.  We  plan  to  use  this  relaxed  matcher  in  our  knowledge  acquisition  tool  for  several 
purposes.  One  is  to  suggest  model  methods  when  the  user  creates  a  new  method  (as  in  the 
find-airport 8  example).  Another  possible  use  is  in  suggesting  concrete  solutions  to  agenda 
errors.  For  example,  if  no  method  is  found  to  achieve  a  goad  the  system  may  suggest  to  use 
a  method  found  by  the  relaxed  matcher  and  indicate  how  to  reduce  their  differences. 

When  EXPECT  checks  the  effects  in  the  knowledge  bases  of  any  change  introduced  by 
the  user,  the  agenda  reflects  any  possible  lapses  that  may  arise.  Each  type  of  item  on  the 
agenda  is  associated  with  a  set  of  possible  remedies  that  the  system  am  suggest  to  the  user 
to  resolve  the  item.  All  of  this  information  is  explicit  in  the  knowledge  acquisition  tool. 
We  plan  to  categorize  in  more  detail  the  possible  lapses  that  may  occur  during  knowledge 
acquisition,  their  relevance  to  the  task  at  hand,  and  the  possible  actions  to  resolve  them.  For 
example,  an  item  that  signals  that  no  method  is  available  to  achieve  a  frequently  occurring 
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goal  is  crucial,  while  an  item  requiring  information  on  a  location  that  is  never  used  may  be 
dismissed  by  the  user.  This  categorization  would  allow  the  user  to  identify  coherent  states 
of  the  knowledge  base  during  the  knowledge  acquisition  process,  when  the  system  is  ready 
for  solving  the  task  and  can  solve  problems  with  an  improved  version  of  the  knowledge  base. 
Lapse  categorization  would  also  allow  better  management  of  the  agenda  through  a  priority 
mechanism  based  on  the  relevance  of  agenda  items. 

8  Conclusion 

We  have  presented  EXPECT,  a  reflective  architecture  for  knowledge  refinement  that  derives 
a  rich  representation  of  the  functionality  of  each  piece  of  knowledge  about  a  task.  The 
knowledge  acquisition  tool  uses  this  functionality  to  reason  about  knowledge  interactions 
and  support  the  user  in  changing  the  knowledge  base.  This  makes  EXPECT  independent  of 
the  problem-solving  method  of  the  task,  a  key  feature  that  distinguishes  our  approach  from 
current  knowledge  acquisition  tools. 
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